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-*  Experimentation  to  develop  a  specialized  reference  key  to  enable  interpreters  to 
avoid  certain  common  errors  in  reporting  information  derived  from  imagery  taken  over 
Vietnam  is  reported  in  the  present  Technical  Research  Note.  The  research  study,  a 
joint  effort  of  personnel  of  the  Behavior  and  Systems  Research  Laboratory  (BESRL)  and 
of  the  North  American  Rockwell  Information  Systems  Company  ( NARISCO),  was  undertaken  to 
construct  and  validate  a  new  type  of  reference  --  an  error  key  --  for  use  in  Vietnam. 
The  key  was  constructed  from  operational  imagery  of  Vietnam  territory  with  annotations 
pointing  out  types  of  error  to  be  avoided.  Evaluation  was  conducted  using  trained 
interpreters  to  determine  if  use  of  the  error  key  would  achieve  the  following  effects 
on  image  interpreter  reports:  1)  increase  number  of  right  responses;  2)  decrease  num¬ 
ber  of  wrong  responses;  and  3)  increase  accuracy  of  identification. 

Performance  results  achieved  by  122  enlisted  image  interpreters  (recent  II  Course 
graduates)  were  used  In  evaluating  effectiveness  of  the  error  key.  Each  of  two  matched 
groups  interpreted  two  sets  of  imagery,  one  set  using  the  error  key,  the  other  without 
using  the  key.  Analysis  of  results  showed  that  I)  the  BESRL  error  key  significantly 
improved  the  accuracy  of  inexperienced  interpreters  in  dealing  with  operational  imagery 
on  Vietnam- -39$  increase  in  rights,  26$  decrease  in  errors  made,  and  43$  increase  in 
accuracy  rate;  2)  the  error  key  significantly  improved  the  completeness  of  reporting  on 
the  subject  imagery;  and  3)  differences  in  performance  as  a  function  of  mission  and 
group  were  evident. 
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FOREWORD 


The  ADVANCED  SURVEILLANCE  SYSTEMS  research  program  of  the  Behavior  and 
Systems  Research  Laboratory  has  as  its  objective  the  production  of  scientific  data  bearing 
on  the  extract,,  n  c*  information  f>-om  surveillance  displays,  and  the  efficient  storage,  retrieval, 
and  transmissk-  of  this  information  within  an  advanced  computerized  :mage  interpretation 
facility.  Research  results  are  used  in  future  systems  design  and  in  the  development  of  en¬ 
hanced  techniques  for  all  phases  of  the  interpretation  process.  Research  is  conducted  under 
Army  RDT&E  Project  2Q662704A721,  "Advanced  Surveillance  Systems,"  FY  1972  Work 
Program.  BESRL  research  in  this  area  is  conducted  as  an  in-house  effort  augumented  by 
contracts  with  organizations  selected  as  having  unique  capabilities  and  facilities  for  research 
in  aerial  surveillance.  The  present  study  was  conducted  jointly  by  personnel  of  the  Behavior 
and  Systems  Research  Laboratory  and  North  American  Rockwell  Information  Systems  Company 
(NABISCO). 

The  BESRL  Work  Sub-unit,  "Influence  of  Displays  on  Image  Interpreter  Performance", 
focuses  on  research  to  determine  how  interpreter  performance  is  affected  by  variations  in  the 
character  of  the  image.  The  present  publication  reports  on  experimentation  to  develop  a 
specialized  reference  key  to  enable  interpreters  to  avoid  certain  common  errors  in  reporting 
information  derived  from  imagery  taken  over  Vietnam. 


EFFECTIVENESS  OF  AN  ERROR  KEY  FOR  IMAGE  INTERPRETATION  IN  VIETNAM 

BRIEF 


Requirement: 

To  construct  and  validate  a  new  type  of  reference~an  error  key--to  help  image  interpreters 
produce  more  accurate  and  complete  interpretations  of  operational  imagery  of  Vietnam. 


Procedure: 

Categories  of  error  to  be  treated  in  an  error  key  were  determined  from  the  interpreter  per¬ 
formance  of  recent  enlisted  graduates  of  the  Army  Image  Interpretation  Course  at  Fort  Holabird. 
A  key  was  constructed  using  operational  imagery  of  Vietnam  similar  to  but  not  identical  with 
that  used  later  to  evaluate  the  key.  Effectiveness  of  the  key  was  evaluated  using  results  achieved 
by  122  enlisted  graduates  of  the  above  course.  Each  of  two  matched  groups  interpreted  two 
sets  of  imagery,  one  set  using  the  error  key,  the  other  without  using  the  key.  Results  were  ana¬ 
lyzed  for  number  of  correct  identifications,  number  of  wrong  identifications,  and  accuracy  of 
identification. 


Findings: 

Practical  and  significant  improvement  with  use  of  the  error  key  was  found-a  39%  increase  in 
tne  number  of  right  responses,  a  26%  decrease  in  the  number  of  errors  made,  and  a  43%  increase 
in  accuracy  rate. 


Utilization  of  Findings: 

Use  of  the  BESfiU  Error  Key  can  improve  significantly  the  accuracy  and  completeness  of 
the  product  of  school  trained  but  inexperienced  U.  S.  Army  interpreters  viewing  operational 
imagery  of  Vietnam.  The  key  can  be  used  to  advantage  in  the  field  in  Vietnam  and  for  training 
in  the  Image  Interpretation  Course. 

The  technique  of  error  avoidance  demonstrated  by  the  key  can  be  extended  both  ir  the 
field  and  in  the  schooi  to  other  imagery  for  different  regions  of  the  world. 


TABLES 


Table  l.  Target  types  involved  in  false  alarms 

2.  Omission  errors  by  mission  and  target  type 

3«  F  values  of  analysis  of  variance 

4.  Key  vs  No-Key  comparisons  -  mean  rights, 
wrongs,  and  accuracy 

5*  Group  comparisons  -  mean  rights,  wrongs , 
and  accuracy 

6.  Mission  comparisons  -  mean  rights,  wrongs* 
and  accuracy 
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EFFECTIVENESS  OF  AN  ERROR  l\EY  FOR  IMAGE  INTERPRETATION  IN  VIETNAM 


BACKGROUND 

Image  interpreters  traditionally  utilize  a  wide  variety  of  informal 
and  formal  refarences  (keys)  in  performing  their  interpretation  tasks. 
Informal  references  may  be  notes,  sketches,  or  annotated  photographs. 
Formal  references  are  pictorial  depictions  of  various  targets  under  vari¬ 
ous  conditions.  These  photointerpretationl^  (Pi)  keys  came  into  existence 
early  in  World  War  II  as  a  result  of  British-sponsored  efforts  in  aerial 
reconnaissance.  In  the  United  States,  production  of  keys  began  as  a 
matter  of  expediency  under  a  great  deal  of  time  pressure.  The  earliest 
keys,  which  illustrated  industrial  facilities  of  various  types,  did  much 
to  set  the  pattern  for  subsequent  keys  produced  in  the  United  States. 

As  requirements  for  identification  of  new  target  categories  evolved, 
military  expediency  still  ruled  out  any  broad  review  of  photographic  in¬ 
terpretation  key  techniques  before  a  given  key  was  prepared  for  use.  Be¬ 
cause  of  extremely  diverse  subject  matter  and  urgent  need,  little  stan¬ 
dardization  in  format,  terminology,  or  technology  was  achieved  during 
this  period. 

The  war  in  Korea  directed  attention  to  the  need  for  updating  and  ex¬ 
panding  keys  to  cover  new  targets  and  target  areas.  Again,  time  pressure 
and  expediency  dictated  many  of  the  ground  rules.  The  period  from  1950 
to  1958  was  one  of  intensified  key  production,  with  interest  concentrated 
on  production,  expeditious  update  schedules,  preplanning  of  keys  and  ways 
of  presenting  keys.  Attention  was  also  given  to  the  formal  layout  and 
approach  to  be  used  in  preparing  keys.  Considerable  standardization  was 
achieved  at  this  time  through  a  joint  8ervice_q>roduct ion  .program.  Most 
keys  were  quite  elaborate,  giving  different  views  and  precise  descriptions 
of  the  targets  in  question  and  their  associated  features,  although  some 
keys  were  quite  simple,  showing  only  outline  sketches  of  targets  to  be 
identified  and  giving  their  length  and  width. 

The  main  purpose  of  a  key  is  to  help  the  interpreter  make  a  more 
accurate  and  complete  identification  of  objects  shown  on  the  imagery. 

It  thus  serves  as  a  standard  for  comparison  of  a  suspect  target  with  a 
known  target.  Keys  are  particularly  useful  when  image  quality  is -mar¬ 
ginal,  when  the  target  to  be  identified  is  atypical  of  its  class,  or  when 
the  target  is  one  seldom  encountered  in  normal  work.  Of  course,  like  any 
other  reference  material,  keys  also  can  be  used  for  training  purposes. 

Since  one  function  of  keys  is  to  reduce  error  in  Interpreter  reports, 
it  was  felt  that  error  might  be  still  further  reduced  by  use  of  an  error 
key  which,  unlike  the  conventional  key,  depicts  commonly  occurring  errors. 


i-' Later  termed  image  interpretation  (il)  keys  to  include  newer  forms  of 
imagery  such  as  infrared  ahd  radar. 


The  error  key  explains  what  the  objects  which  resemble  the  target  really 
are  and  how  they  differ  from  the  target.  The  conventional  key  makes  no 
mention  of  the  possibility  that  other  objects  may  look  like  certain  tar¬ 
gets  but  only  discusses  attributes  of  actual  targets.  The  difference  be¬ 
tween  the  two  keys  is  illustrated  by  the  following  statements  about 
weapons  positions: 

Conventional  Key.  Two  open  positions  joined  by  connecting  trenches 
with  clear  fields  of  fire.  .  . 

Error  Key.  The  items  at  (l)  are  not  weapons  positions  but  are 
graves  with  walls  surrounding  the  grave  mound  with  a  path  from 
village  to  grave  site.  Poor  tactical  position  (such  as  no  field 
of  fire)  is  clearly  indicated. 

An  earlier  study?!' ,  based  on  World  War  II  imagery,  demonstrated  the 
value  of  an  error  key.  Use  of  an  error  key  resulted  in  an  operationally 
useful  and  statistically  significant  decrease  In  errors  in  interpretation 
reports  concerning  convoys  and  artillery  positions.  Since  the  key  used 
was  designed  to  reduce  false  alarms  (reporting  a  non-military  object  as 
one  of  military  significance),  its  use  did  not  affect  the  number  of  right 
identifications  made. 

An  error  key  approach  seemed  particularly  appropriate  to  the  South¬ 
east  Asia  situation  because  of  the  likelihood  of  confusion  because  of 
the  similarity  of  military  and  civilian  enemy  activities.  Moreover,  after 
an  analysis  of  image  interpretation  reports  of  inexperienced  image  inter¬ 
preters  in  Vietnam,  it  became  apparent  that,  in  addition  to  a  need  for 
reduction  of  false  alarms,  common  errors  of  omission  (not  reporting  a 
military  target)  were  also  a  subject  for  error  reduction. 


PURPOSE 

The  objectives  of  this  effort  were  to  construct  and  evaluate  an 
error  key  for  use  in  Vietnam.  The  key  was  to  be  constructed  from  opera¬ 
tional  imagery  of  Vietnam  territory  with  annotations  pointing  out  types 
of  error  to  be  avoided.  The  evaluation  was  conducted  using  trained  inter¬ 
preters  to  determine  if  use  of  the  error  key  would  achieve  the  following 
effects  on  interpreter  reports:  1)  increase  rights,  2)  decrease  wrongs, 
and  3)  increase  accuracy. 


S-'Martinek,  Harold,  and  Robert  Sadacca.  Error  keys  as  reference  aids  in 
image  interpretation.  Technical  Research  Note  153*  Behavior  and 
Systems  Research  Laboratory,  Arlington,  VA.  June  1963. 
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CONSTRUCTION  OF  EftiWR  KEY  ■ 

i 

In  the  first  phase  of  construction  of  the  error  key,  imagery  taken 
over  Vietnam  territory  was  interpreted  by  50  recent  graduates  of  the  en¬ 
listed  Image  Interpretation  Course  at  Fort  Holabird,  and  their  interpre¬ 
tations  were  analyzed  in  Order  to  define  errors.  In  the  second,  phase, 
imagery  selected  as  representative,  based  on  the  analysis  of  typical 
errors,  was  assembled  into  a  key  and  Instructions  for. key  use  were  pre¬ 
pared  . 

Error  Definition 

To  identify  erroru  typically  made  by  image  interpreters,  imagery  was 
selected  from  two  missions.  MSN  5536  covers  an  area  of  Vietnam  located 
approximately  40  miles  northwest  of  Saigon.  The  terrain  is  mostly  flat 
with  meandering  streams.  Nearly  two-thirds  of  the  area  is  either  wooded 
or  abandoned  agricultural  land.  The  remainder  is  actively  cultivated  in 
tea  and  row  crops.  There  are  two  urban  centers.  The  other  mission,  MSN 
6558,  covers  an  area  located  15  to  20  miles  southwest  of  Saigon.  This 
area  is  intensively  cultivated,  with  rice  as  the  principal  crop.  Some 
trees  are  found  in  the  hamlets  and  bordering  streams.  The  Yam  Go  River 
flows  through  a  corner  of  thi3  ares.  The  approximate  scale  of  both  mis¬ 
sions  is  1:5000.  The  missions  consist  of  prints  in  9"  x  18"  format. 
Thirteen  prints  were  used  from  MSN  5536  and  19  from  MSN  6558*  Interpre¬ 
tation  reports  were  collected  following  the  procedure  described  in 
Appendix  A,  Instructions  to  the  Examinee ♦  The  examinees  used  the  same 
target  list  (Appendix  C)  and  report  form  (Appendix  D)  as  were  later  used 
in  the  evaluation  experiment.  Each  interpreter  had  available  stereoscopes 
(2X  and  4X),  and  8X  tube  magnifier,  and  a  slide  rule.  .Objects  found  by 
the  interpreters  were  annotated  by  number  in  grease  pencil  directly  on 
the  paper  prints.  This  procedure  avoided  the  possibility  of  errors  in 
reading  and  transcribing  coordinates.  The  description  of  the  object  found 
was  recorded  on  a  special  target  reporting  form.  Each  response  was  re¬ 
corded  on  a  master  set  of-  imagery  to  allow  analysis  to  determine  what 
Image  characteristics  tended  to  attract  interpreters’  attention  most  often. 
Tabulation  of  responses  was  then  analyzed  for  errors  of  two  types,  false 
alarms  and  omissions.  Errors  were  classified,  first  by  false  response 
and  then  by  the  non-military  object  or  terrain  feature  believed  to  have 
caused  the  error. 

i 

Table  1  shows  the  false  alarm  targets  classified  into  eight  categories 
weapons  positions,  vehicles,  sampans,  etc.  Objects  giving  rise  to  false 
alarms  included  graves,  craters,  brush,  and  trees.  The  two  image  sets 
produced  similar  but  not  identical  errors.  For  example,  10$  of  the  false 
alarms  from  MSN  655®  were  reports  of  mine  fields  but  jjQjnine  fields  were 
reported  on  MSN  553 6.  For  MSN  5536,  total  false  alarms  were  219  with  87$ 
classifiable;  for  MSN  655®,  there  were  536  false  alarms  with  96$  classi¬ 
fiable. 

Table  2  presents  the  omission  errors  by  Mission  and  target  type. 

The  first  column  under  each  mission  presents  the  number  of  targets  present 
in  that  mission  for  each  of  twelve  target  types.  Comparison  of  these 


55*  *-.-»:  aw 


Table  1 


TARGET  TYPES  INVOLVED  IN  FALSE  ALARMS 


False  Alarm 


Weapons  position 


Vehicles 


Sampans 
Supply  points 


Personnel 


Tunnel  entrances 


Mine  fields 


Bunkers 


Object  Giving  Rise  to 
_ False  Alarms 

Graves 

Craters 

Brush  and  trees  (ground) 
Brush  and  flotsam  (rivers) 


Straw 


Wells 


Is  sion  635c 
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figures  indicates  a  difference  between  missions  in  number  and  kind  of 
targets  present.  Column  2  for  each  mission  gives  the  percentage  of 
errors  of  omission  for  each  target  type  and  is  an  index  of  the  difficulty 
of  finding  each  target  type  in  this  imagery.  Column  3  gives  the  percent¬ 
age  of  total  omissions  and  is  an  index  of  how  much  impact  this  error  of 
omission  will  have  on  the  total  report  of  the  mission.  For  example,  a 
high  percentage  in  column  2  could  have  little  effect  if  column  1  indi¬ 
cated  that  only  one  target  was  present.  The  percent  omitted  values 
(column  2)  were  obtained  by  summing  separately  for  each  target  the  num¬ 
ber  of  omissions  for  all  examinees,  and  then  dividing  this  sum  by  the 
.  product  of  the  frequency  of  occurrence  of  the  target  and  the  number  of 

,4  examinees.  The  percent  of  total  omissions  values  (column  3)  were  obtained 

i  by  again  summing,  separately  for  each  target,  the  total  number  of  omis- 

*  sions  for  all  examinees,  then  dividing  the  sum  by  the  total  number  of 

omissions  for  all  targets  by  all  examinees. 


Based  on  these  results,  vehicles,  sampans,  and  personnel  were  se¬ 
lected  as  the  targets  to  be  emphasized  in  the  error  key  because  of  their 
high  difficulty  (column  2)  and  high  impact  on  the  interpreter's  report 
(column  3).  Weapons  positions  were  initially  included  as  a  target  type 
to  be  considered,  but  were  rejected  because  of  disparity  in  results  be¬ 
tween  the  two  missions.  Also,  it  was  felt  that  added  emphasis  on  this 
type  of  target  might  increase  one  of  the  severest  problems  in  false  j 

alarms  reporting  (Weapons  positions  was  the  largest  error  category  in  j 

Table  1.)  i 

„  i 

In  addition  to  the  above  analyses,  high  pan  imagery  from  MSN  p037>  j 

scale  1:8000,  covering  more  rugged  terrain  with  heavy  forest  cover.,  was  l 

analyzed  in  a  similar  procedure,  although  much  less  completely.  Similar 
but  not  identical  error  types  were  found  and  included  in  the  final  error 
key,  to  provide  a  more  general  key.  However,  high  pan  imagery  was  not  t 

included  ir.  the  evaluation  of  the  key.  The  additional  error  typ’es  found  j 

were  identified  as  slow-down  obstacles,  mud  puddles,  trails,  and  ditches.  ! 


Error  Key  Preparation 

The  error  key  consists  of  three  pages,  two  pages  with  two  stereo 
pairs  on  each  and  one  page  with  four  non-stereo  images.  The  stereo  pairs 
of  the  error  key  were  also  prepared  from  imagery  taken  from  MSN  555$  and 
MSN  6358*  but  did  not  include  imagery  frames  used  in  the  study.  The  non¬ 
stereo  chips  were  taken  from  MSN  8037* 

From  three  to  nine  objects  were  annotated  on  each  illustration.  A 
brief  description  of  each  annotation  was  included.  A  more  thorough  dis¬ 
cussion  was  presented  in  an  attachment  sheet  which  consisted  of  no  more 
than  a  four-sentence  description  of  an  individual  annotation.  Descrip¬ 
tions  were  presented  in  two  basic  formats,  a  cautionary  note  type  and  a 
not  type.  These  two  types  of  statement  were  intermingled  in  the  narra¬ 
tive  as  appropriate  to  provide  variety.  Below  are  examples. 
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Cautionary  Note.  Annotation  (1)  shows  walled  graves  of  a  special 
type  frequently  misidentified  as  vehicles.  The  length-width  ratio, 
square  corners,  and  flat  appearance  inside  the  low  walls  can  be 
very  deceptive,  particularly  when  a  few  graves  of  this  type  are 
found  in  isolation.  Noting  the  lack  of  clearly  associated  trackage 
together  with  careful  measurement  will  identify  these  objects  as 
graves. 

Not  Statement.  The  items  at  (1)  are  not  weapons  positions  but 
mound  graves  which  are  sometimes  found  in  groups,  with  the  older 
ones  almost  completely  overgrown.  General  diagnostic  features  of 
the  graves — random  arrangement,  absence  of  trackage,  size,  siting, 
etc. --should  be  noted  in  making  identifications.  Annotation  (2) 
indicates  not  foxholes,  but  wells  of  a  certain  type  dug  into  the 
dikes  between  fields.  Wells  are  usually  old  and  regularly  spaced 
along  the  entire  length  of  the  dike.  Foxholes  and  weapons  positions 
are  usually  more  numerous  and  freshly  dug  and  not  as  regularly 
spaced. 

The  verbal  portion  of  the  key  was  read  slowly  to  the  examinees  with 
appropriate  pauses  to  give  them  sufficient  time  to  study  the  examples 
shown  on  the  Imagery.  The  examinees  then  were  allowed  to  keep  the  error 
key  at  their  work  stations  to  use  during  the  testing.  Appendix  B  contains 
“he  descriptive  material  of  the  Error  Key. 


EVALUATION  OF  ERROR  KEY 

The  general  procedures  for  the  evaluation  experimentation  was  to 
have  a  new  sample  of  interpreters  extract  information  from  photographs 
with  and  without  use  of  the  error  key.  In  addition  to  the  test  imagery 
(which  was  the  same  as  that  used  for  error  definition)  and  the  error  key, 
examinees  were  given  a  Southeast  Asia  target  lict  (Appendix  C)  and  an 
interpretation  report  form.^  Appendix  D) .  The  target  list  established  in¬ 
formation  requirements  to  which  the  interpreter  was  to  respond.  The 
first  section  presented  a  l^st  of  specific  items  to  be  annotated  on  the 
imagery  such  as  sampans  and  weapons  positions.  The  second  was  a  Remarks 
section  requiring  reports  of  three  types  as  appropriate,  but  no  annota¬ 
tions.  These  reports  dealt  with  the  approximate  number  of  foxholes, 
amount  of  trail  activity,  and  number  of  road  cuts  to  be  found  in  an  in¬ 
dividual  photo. 

The  answer  sheet  provided  space  for  the  target  entry  and  associated 
remarks  in  senarate  columns.  The  associated  remarks  could  provide  such 
information  as:  ’’Sampan  is  motorized".  "Personnel  plowing  or  gathering 
crops."  "Object  is  one-man  foot  bridge."  Space  was  also  provided  for 
an  estimate  of  confidence  attached  to  each  target  located,  using  the 
levels  Certain,  Probable,  Possible. 


-  7 


n  r  1 1  ^  '1  —  ^  ■ 


fi 


Sample  Used  in  Evaluating  Key 

The  image  interpreters  used  in  evaluating  the  error  key  were  122 
enlisted  men,  recent  graduates  from  the  Image  Interpretation  Course  at 
Fort  Holabird,  Maryland.  During  their  training,  they  had  received  more 
than  eight  hours  of  intensive  instruction  on  Southeast  Asia  targets. 
Training  included  use  of  a  conventional  key  for  interpreting  imagery 
obtained  in  insurgency  and  counter-insurgency  situations.  Additional 
training  on  Southeast  Asia  targets  was  provided  in  various  parts  of  the 
course  concerned  with  other  topics.  For  example,  in  training  on  the 
interpretation  of  infrared  imagery,  at  least  some  of  the  exar-"les  pre¬ 
sented  were  taken  in  Vietnam  and  showed  typical  Southeast  Asia  targets. 


.Experimental  Design 

A  2  x  2  x  2  Latin  square  design  with  replication  was  used,  each 
examinee  serving  as  his  own  control.  This  design  allowed  for  a  relatively 
sensitive  test  of  the  main  variable  ( interpretation  with  key  versus  with¬ 
out  key).  The  independent  variables  were  two  sets  of  imagery  (Missions 
5536  and  635^),  two  groups  of  matched  examinees,  and  the  two  key  condi¬ 
tions.  Design  is  shown  below: 


Day  I  Day  II 

Without  Key  With  Key 

Group  A  MSN  5536  MSN  6358 

Group  B  MSN  6358  MSN  5536 

A  possible  source  of  confounding  occurred  because  the  test  with  use 

of  key  was  always .administered  after  the  no-key  test.  Thus,  examinees 
under  the  key  condition  were  exposed  to  more  Vietnamese  imagery,  having 
previously  been  engaged  in  interpretation  under  the  no-key  condition. 
This  effect  was  believed  to  be  minimal  because  there  was  no  feedback  to 
the  examinees  on  their  work  under  the  no-key  condition  and  thus  little 

if  any  learning.  Moreover,  different  imagery  was  used  for  the  two  con¬ 

ditions. 


Dependent  Variables 

Rights  Score.  Number  of  correct  identifications.  This  score  is 
perfectly  correlated  with  completeness,  the  latter  defined  as  the  number 
of  right  responses  divided  by  the  number  of  targets  present  in  the 
imagery.  Consequently,  completeness  and  number  of  correct  identifica¬ 
tions  can  be  used  interchangeably  in  statistical  analysis.  Completeness 
of  a  report  is  important  in  that  it  is  a  measure  of  how  many  targets  are 
unreported  (errors  of  omission),  in  other  words,  the  "unknowns"  a  com¬ 
mander  has  to  consider. 
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Wrongs  Score.  Number  of  incorrect  identifications  (errors  of 
commission) .  This  score  gives  an  indication  of  the  number  of  erroneous 
targets  an  interpreter  reports.  Wrong  interpretations  can  be  of  two 
types : 

1.  Misidentifications  where  a  militarily  significant  target  (truck, 
tank,  etc.)  is  identified  as  some  other  military  target. 

2.  False  alarms  where  a  non-military  object  (brush,  logs,  a  well, 
a  shrine,  etc.)  is  interpreted  as  a  militarily  significant  target  (tank, 
dugout,  AA  position,  bunker,  etc.).  Either  of  these  types  may  cause  the 
commander  using  the  intelligence  information  to  have  a  wrong  estimate  of 
enemy  capability  and  possibly  make  an  incorrect  commitment. 

Accuracy  Score.  Number  of  rights  divided  by  rights  plus  wrongs. 

This  score  informs  the  commander  of  the  number  of  true  targets  relative 
to  the  total  number  reported,  and  consequently  how  much  faith  he  can 
have  in  the  report. 

RESULTS 

Effect  of  Error  Key 

Rights.  The  F  values  in  Table  3  show  that  the  mean  difference  in 
the  rights  scores  between  using  the  error  key  and  not  using  the  error  key 
was  significant  at  the  .05  level.  Table  4  shows  that  the  total  number  of 
correct  responses  increased  from  a  mean  of  25. 05  for  the  no-key  group  to 
a  mean  of  34.88  for  the  group  using  the  key,  an  increase  of  39 $•  Thus, 
use  of  the  error  key  resulted  is  a  statistically  significant  and  opera¬ 
tionally  useful  increase  in  the  number  of  correct  targets  the  interpreters 
found — in  other  words,  a  useful  increase  in  the  completeness  of  the  re¬ 
port.  The  complete  analysis  of  variance  tables  are  given  in  Appendix  E. 

Wrongs.  Table  3  also  shows  that  the  mean  difference  in  the  wrongs 
score  between  the  key/no-key  conditions  was  significant  at  the  .05  level. 
From  Table  4,  the  number  of  incorrect  responses  decreased  from  a  mean  of ' 
45.6  in  the  no-key  group  to  a  mean  of  33>8  for  the  key  group,  a  decrease 
of  26$.  Thus,  not  only  does  use  of  the  error  key  increase  correct  re¬ 
sponses  but  it  also  results  in  a  statistically  significant  and  operation¬ 
ally  useful  decrease  in  the  number  of  wrong  identifications  an  interpre¬ 
ter  reports. 

Accuracy.  The  mean  difference  in  accuracy  was  significant  at  the 
.05  level  (Table  5)»  Since  use  of  the  error  key  both  increased  the 
number  of  rights  and  decreased  the  number  of  wrongs,  it  was  expected  that 
accuracy  would  be  substantially  greater  with  the  key  than  without.  In 
fact,  mean  accuracy  increased  from  35  t0  5*  (43$)  with  use  of  the  key. 

Use  of  the  error  key  thus  produces  a  statistically  significant  and  opera¬ 
tionally  useful  increase  in  the  accuracy  rate  of  interpretation. 
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Table  5 


GROUP  COMPARISONS  -  MEAN  RIGHTS,  WRONGS, 
AND  ACCURACY 


Group  A 
Group  B 


Mean 

Mean 

Mean 

Rights 

Wrongs 

Accuracy 

35*1 

42,8 

43.7 

26.8 

36.6 

42.3 

Table  6 

MISSION  COMPARISONS  -  MEAN  RIGHTS,  WRONGS, 
AND  ACCURACY 


Rights 

Wrongs 

Accuracy 

i 

\ 

MSN  5536 

26.2 

33-8 

43.7 

V 

\ 

•1 

MSN  6358 

33-7 

45.6 

42.5 

?! 

Group  and  Mission  Effects.  Analysis  of  the  effectiveness  of  the 
Error  Key  In  terms  of  rights,  wrongs,  and  accuracy,  was  the  primary 
objective  of  the  experiment.  However,  control  or.  groups  and  mission 
from  which  the  imagery  was  taken  was  included  in  the  analysis  to  increase 
precision  of  results.  Results  of  the  analysis  of  groups  and  missions 
were  desired  as  indication  of  the  need  for  control  on  these  variables  In 
future  experiments.  Table  3  shows  that  the  mean  difference  between  groups 
was  statistically  significant  only  for  the  Rights  score.  From  Table  5/ 
Group  A  had  24$  more  rights  than  Group  B.  Future  experimentation  should 
therefore  use  counterbalancing  techniques  in  addition  to  matching  tech¬ 
niques  for  control  of  examinee  differences. 


at. 


Mean  differences  between  missions  were  significant  at  the  .05  level 
for  rights  and  for  wrongs,  but  not  significantly  different  for  accuracy 
(Table  5)*  From  Table  6,  MSN  635®  had  28$  more  rights  and  35$  more 
wrongs  than  MSN  5556.  Reports  on  MSN  6358  had  a  mean  completeness  of 
only  6$  compared  to  23$  for  MSN  5536.  However,  these  figures  must  be 
interpreted  with  caution  because,  although  equal  time  was  allowed  for 
interpretation  of  the  missions,  the  test  imagery  from  MSN  6538  consisted 
of  15  frames  with  637  targets  whereas  only  13  frames  with  115  targets 
were  presented  from  MSN  5536. 


CONCLUSIONS 

Although  implications  of  the  findings  from  the  present  study  can 
have  only  limited  generalizability  for  Vietnam  because  of  a  multitude  of 
uncontrolled  variables--experience,  scale  of  imagery,  type  of  report, 
time  allowed,  etc . --results  of  a  specific  study  such  as  this,  based  on 
reliable  data,  are  important  as  guidelines  in  establishing  a  baseline  for 
training  and  utilization  of  interpreters.  Average  completeness  was  only 
7$  for  the  no-key  experimental  condition,  which  may  be  representative  of 
image  interpreter  performance  for  a  novice  going  to  Vietnam  directly  from 
the  Image  Interpretation  Course  and  not  yet  immersed  in  the  specific 
military  situation  nor  having  gained  field  experience.  Even  with  use  of 
the  error  key,  completeness  of  interpretation  was  only  10$.  The  need  for 
improved  operational  performance  indicated  l-y  these  results  is  reflected 
in  the  establishment  of  the  present  on-the-job  training  programs  in 
Vietnam.  Unfortunately,  this  training  reduces  the  productive  time  in 
Vietnam  during  the  one-year  tour  of  duty. 

A  second  major  consideration  is  the  level  of  accuracy  of  the  reports. 
Mean  accuracy  of  about  35$  was  found  before  use  of  the  key.  Even  the  51$ 
accuracy  achieved  with  the  key  means  that  half  the  information  provided 
is  wrong,  leaving  considerable  room  for  improvement  and  again  indicating 
need  for  additional  training,  perhaps  on  the  job.  Training  should  include 
opportunity  for  feedback  to  the  interpreter  so  that  he  can  determine 
whether  or  not  he  was  right  or  wrong  in  his  report  and  learn  by  his  mis¬ 
takes.  In  this  context,  it  is  apparent  that  there  is  continued  need  for 
implementation  of  new  team  approaches  and  other  interpreter  techniques  to 
bring  about  improvement. 


SUMMARY  OF  RESULTS 

The  bfiSRL  Error  Key  significantly  improved  the  accuracy  of  inexperi¬ 
enced  interpreters  in  dealing  with  operational  imagery  on  Vietnam. 

The  Error  Key  significantly  improved  the  completeness  of  reporting 
on  imagery  of  Vietnam. 

Differences  in  performance  as  a  function  of  mission  and  group  were 
evident. 
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There  la  considerable  room  for  improvement  in  interpreter  performance 
even  with  the  error  key. 

OPERATIONAL  IMPLICATIONS 

The  present  key  should- be  made  operational. 

For  optimal  operational  use,  the  present  key  should'  be  enlarged  to 
include  different  areas  af  Vietnam,  different  scales  and  types  of  photog- 
.raphy,  and  different  sensors  such  as  infrared. 

Other  geographical  areas  of  potential  interest  should  be  considered 
as  topics  for  an  error  key. 

Consideration  should  be  given  to  incorporating  more  craining  in 
error  avoidance  into  the  curriculum  at  the  Intelligence  School. 

Error  analysis  should  bo  applied  to  performance  in  t,he  image  inter¬ 
pretation  course  and  on-fche-jcL  training  to  help  define  areas  which  need 
improvement . 
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APPENDIX  A 


INSTRUCTIONS  TO  THE  EXAMINEE 

A.  Today  we  have  a  research  project  using  imagery  representative  of 
operational  imagery  used  in  Vietnam. 

This'  research,  w'.  Ich  we  call  an  error  study,  is  different  from  most 
research  in  that  we  are  most  interested  in  what  you  do  wrong.  We  are 
hoping  that  by  analysis  of  the  special  mistakes  and  the  general  types  of 
errors  made  by  you  and  the  other  classes  who  have  worked  with  this 
imagery,  we  can  develop  methods  for  improving  II  keys. 

As  you  may  know,  there  is  a  DIA  key  (Insurgency  and  Counter-insur¬ 
gency  in  Vietnam)  dealing  with  Vietnamese  imagery,  and  the  1st  MI  Bn  (ARS) 
made  a  key  for  use  in  Vietnam.  We  are  hoping  that,  from  data  determined 
by  the  research  we  conduct  here,  we  will  be  able  to  develop  guidelines 
for  producing  better  keys,  including  supplemental  keys  in  the  field.  We 
have  several  studies  on  scales,  the  type  of  perspective,  use  of  drawings, 
and  other  aspects  of  how  to  put _an  .II  key  together.  Today’s  research  is 
concerned  not  with  how  to  format  the  key,  but  rather  what  to  put  in  it. 

By  asking  you  to  interpret  Vietnamese  imagery  with  relatively  little 
experience  with  the  country  and  insurgency  targets,  we  know  you  will  make 
mistakes.  The  most  common  mistakes  are,  naturally,  the  topics  which  are 
most  important  for  inclusion  in  a  key.  Thus,  we  call  this  an  Error  Study. 
By  studying  the  errors  which  you  make,  we  hope  to  be  able  to  develop  more 
efficient  and  effective  keys  oriented  specifically  to  avoiding  the  common 
pitfalls  in  identification  of  COIN  type  targets. 

B.  TARGET  LIST 

The  target  list  for  use  in  this  study  is  divided  into  two  sections; 
a  list  of  specific  items  to  annotate,  such  as  sampans,  weapons  positions, 
etc.,  and  three  types  of  information  to  report  when  appropriate,  without 
requiring  annotations.  These  three  items,  foxholes,  road  cuts,  and  trail 
activity,  are  discussed  on  the  bottom  of  your  target  list.  After  you 
have  looked  at  some  of  this  imagery,  it  will  be  obvious  that  for  many 
areas  the  counting  and  reporting  of  individual  foxholes  is  a  very  tedious 
task,  thus,  we  do  not  require  you  to  annotate  them  but  just  to  report  the 
approximate  number  on  the  frame.  Road  cuts  and  trails  are  also  found  in 
quantity  and  need  not  be  annotated  individually. 

Urban  areas  may  be  ignored  completely,  as  obviously  there  will  be 
too  many  personnel,  vehicles,  bridge*,  etc.  to  worry  about  in  these  areas. 
Urban  areas  where  they  occur  will  be  designated  on  your  prints. 

Note  that  "weapons  positions”  includes  covered  and  fortified  posi¬ 
tions,  as  well  as  open  pits  and  firing  bays  along  trenches,  etc. 


Preceding  page  blank 
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The  targets  which  you  do  find  are  to  be  annotated  with  wax  pencils 
on  the  paper  prints  and  reported  on  the  forms  provided.  The  annotations 
should  be  made  by  neatly  outlining  the  target  or  target  complex  and  then 
numbering  the  annotation.  Where  items  occur  close  together,  it  may  be 
desirable  to  tie  the  number  to  the  proper  annotation  with  a  "tick  mark." 
Do  not  use  large  circles  for  annotations  as  they  may  include  more  targets 
than  you  report. 
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C.  REPORTING  FORM 

The  reporting  form  we  will  use  for  this  test  is  fairly  self-explan¬ 
atory. 

Mission  number  is  the  mission  number  from  your  envelope  of  imagery 
and  the  copy  number--for  instance,  MSN  5536  Copy  19. 

Frame  number  is  indicated  in  the  titling  strip  on  each  frame. 

Please  use  a  separate  form  for  each  frame  on  which  you  report  targets  so 
that  you  can  go  back  for  additional  items. 

II  name  Is  your  name. 

Annotation  number  (col  1).  You  may  annotate  as  you  choose,  but  it 
will  probably  be  easier  to  start  over  with  number  1  on  each  frame. 

Identification  (col  2).  It  is  important  that  you  use  the  terms  on 
the  target  list. 

Quantity  (col  5).  Indicate  quantities  of  weapons  positions,  person¬ 
nel,  etc.  in  this  column. 

> 

Confidence  ( col  4) .  Indicate  certain,  probable,  or  possible,  or 
some  reasonable  abbreviation  thereof,  for  each  target  reported. 

Remarks  (col  5).  The  last  column  can  be  used  for  remarks  about  the 
target,  the  annotations,  and  any  other  comments  you  may  wish  to  make. 

The  bottom  three  lines  are  already  labeled  with  foxholes,  road  cuts 
and  trails.  Fill  in  confidence  and  quantities,  etc.,  as  indicated  in  the 
target  list. 

D.  INTERPRETATION  LOGGING  FORM 

In  addition  to  cur  main  research,  we  are  also  interested  in  the 
amount  of  time  required  to  work  with  this  type  of  imagery.  We  have  pre¬ 
pared  a  logging  form  to  record  the  time  as  you  go  along.  The  most  effi¬ 
cient  way  is  to  tape  the  form  down  at  the  edge  of  your  table.  Please 
enter  the  clock  time  when  you  start,  and  each  time  you  pick  up  a  new 
print  or  go  back  to  an  old  one.  Enter  the  time  and  the  frame  number  in 
the  spaces  provided.  Record  also  the  time  when  you  start  breaks  and  when 
you  start  to  interpret  again.  From  these  data  we  hope  to  determine  an 
average  time  per  frame  for  various  types  of  imagery  and  various  target 
densities,  for  II 's  of  your  experience  level. 
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We  do  ask  that  you  work  thoroughly  and  carefully.  'In  all  other 
classes,  working  with  this  Vietnamese  imagery,  the  interpreters  rushed 
through  their  interpretations  and  reported  less  than  10#  of  the  actual 
targets  present.  You  will  have  three  hours  of  working  time,  plus  a 
half-hour  break.  From  time  to  time,  we  will  tell  you  how  many  frames 
you  should  have  completed  so  you  can  adjust  your  work  pace  as  necessary.  1 

E.  MISSION  5536  .  ' 

! 

This  mission  covers  an  area  located  approximately  40  miles  northwest 
of  Saigon.  You  will  have  13  9  x  l8  prints  of  the  left  split  of  a  'split- 
vertical  mission  flown  at  approximately  1:5>000  scale.  You  will  have 
three  hours  to  work  with  this  imagery.  You  may  take'  breaks  as  you  desire, 
but  do  record  the  time  oh  your  logging) sheet. 

F.  MISSION  6358  i  «  ' 

You  will  have  19  9  x  18  prints  from  this  split-vertical  mission. 

The  area  is  located  15  to  20  miles  southwest  of  Saigbn  and  is  mostly 
agricultural.  Scale  is  approximately  1:5,000.  You  will  have  three  hourq 
to  report  targets  from  this  mission,  which  is  sufficient  time  to  allow  a 
thorough  study  of  all  frames.  Take  breaks  as  you  see  fit,  but  b(e  sure  to 
log  the  time. 
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VIETNAM  ERROR  KEY  -  1 


Top  Figure 

Annotation  (1)  shows  a  special  type  of  walled  graves  which  are  frequently 
Kiisidenfcified  as  vehicles.  The  length-width  ratio,  square  corners,  and 
flat  appearance  inside  the  low  walls  can  be  very  deceptive  when  a  few 
graves  of  this  type  are  found  in  isolation.  Noting  the  lack  of  clearly 
associated  trackage  and  careful  measurement  will  indicate  these  objects 
to  be  graves. 

At  (2)  are  some  wells  of  a  type  of te .  mistaken  for  weapons  positions. 

The  lack  of  trackage,  lack  of  association  with  other  military  activities, 
and  absence  of  any  military  reason  for  being  where  they  are,  are  often 
the  best  features  for  distinguishing  these  wells  from  military  emplace¬ 
ments. 


Personnel  are  often  difficult  to  detect  at  this  scale  (1:5000).  Their 
movement  between  exposure  of  a  stereo  pair,  coupled  with  their  small 
size,  is  often  the  best  means  of  detecting  them.  Sometimes  shadow  or 
logical  location  are  also  helpful  clues.  At  (5)  are  personnel  along  a 
road  and  in  the  courtyard  of  a  building. 

The  re'd  annotation  (4)  indicates  some  slow-down  obstacles  on  a  road. 

These  serve  the  sane  function  as  road  cuts  but  are  built  up  on  the  top 
of  the  road.  Note  that  these  items  at  (4)  are  not  vehicles,  even  though 
they  have  the  same  general  shape  and  size.  Indicators  of  their  true 
nature  are  their  staggered  pattern  on  opposite  sides  of  the  road  and  the 
manner  in  which  the  traffic  marks  tend  to  go  around  them. 

Bottom  Figure 

This  illustration  shows  cultural  activities  in  more  open  country.  The 
items  at  (1)  are  not  weapons  positions  but  a  type  of  mound  grave  which 
is  sometimes  found  in  groups,  the  older  ones  almost  completely  overgrown. 
General  diagnostic  features  of  graves — random  arrangement,  absence  of 
trackage,  size,  siting,  etc. --should  be  noted  in  making  identifications. 

Annotation  (2)  indicates  not  foxholes,  but  a  type  of  well  dug  )Lnto  the 
dikes  between  fields.  Wells  are  usually  old  and  regularly  spaced  along 
the  entire  length  of  the  dike.  Foxholes  and  weapons  positions  are 
usually  more  numerous  and  freshly  dug. 

At  (5)  are  wells  out  in  the  fields,  which  illustrate  the  tendency  to  be 
central  to  the  fields  even  more  clearly  than  those  in  the  top  figure. 
Non-strategic  'location,  lack  of  crackage,  etc.  are  further  evidence 
that  these  are  neither  weapons  positions  nor  foxholes. 


Preceding  page  blank 


Personnel  such  as  those  at  (4),  are  always  difficult  to  detect  at  this 
scale. 

Carts,  as  at  (5),  should  be  easily  detected  on  roads  and  somewhat  so  in 
fields,  particularly  when  hitched  to  oxen  and  moving.  Small  size,  lack 
of  regular  shape  when  loaded,  and  lack  of  tonal  contrast  with  many  back¬ 
grounds,  apparently  cause  many  interpreters  to  overlook  these  items. 

In  general,  extra  care  must  be  taken  to  avoid  missing  such  small  items 
as  personnel  and  carts. 

At  (6)  in  red  are  some  mudpuddles  which  are  not  vehicles.  Note  the  lack 
of  sharp  outline  or  square  corners  and  in  particular  the  way  the  pathways 
skirt  these  spots,  indicating  that  they  have  been  present  for  some  time. 
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Thd  rectangular  grave  mounds  at  (1)  can  be  distinguished  from  weapons 
pits  by  the  fact  that  the  shadow  indicates  their  mounded  nature.  The 
lack  of  trackage  or  weapons  positions  precludes  their  being  fortifica¬ 
tions  despite  appearance  of  substantial  concrete  construction.  The 
number  and  siting  are  other  clues  to  consider.  Note  also,  how  some  grave 
mounds  at  the  very  top  of  the  frame  are  completely  overgrown  with  brush 
which  may  look  like  camouflage. 

Often  an  experienced  interpreter  may  mistake  a  round  earth  scar  for  a 
weapons  position.  The  irregular  size,  pattern,  and  lack  of  trackage  or 
any  distinct  shadow  help  identify  the  features  at  (2)  as  craters,  not 
weapons  positions. 

At  (3),  the  mine  field  pattern  results  from  stacking  grain  in  shocks. 

Note  the  lack  of  strategic  value  to  the  location  and  the  slight  irregu¬ 
larity  in  the  size  and  arrangement  of  the  spots.  Considering  that  this 
is  1:5,000  scale  imagery,  the  spots  are  very  large  for  mines. 

Many  things  are  found  in  the  water  other  than  sampans.  Logs,  brush,  and 
garbage  float  down  the  streams,  rocks  protrude  in  spots,  and  tree  shadows 
are  frequently  present  along  the  margins.  At  (4)  are  indicated  tree 
shadows  and  a  log.  Actual  sampans  are  usually  clearly  identifiable  when 
present. 

At  (5)  are  found  a  sampan  in  the  water  and  one  along  side  the  river  bank. 
Why  sampans  are  frequently  omitted  on  II  reports  is  not  clearly  under¬ 
stood. 

Personnel  such  as  those  at  (6)  are  best  identified  by  their  small  size, 
movement  between  exposures,  and  association  with  cultural  activities 
such  as  houses  or  crops. 

Carts  may  also  be  detected  by  movement  or  association  with  transportation 
routes  or,  as  in  (7)>  agricultural  activities. 

At  (8),  the  clustered  dark  objects  are  not  personnel  but  bunches  of  rice 
seedlings  that  have  been  collected  together  in  bunches  in  the  seedbpd 
ready  for  transfer  to  the  paddy  below.  Note  that  these  riqe  bunches  are 
too  blocky  in  appearance  to  be  personnel,  cast  no  noticeable  shadow  be¬ 
cause  they  are  so  short,  lack  the  white  spot  of  a  coolie  hat,  show  no 
motion  in  stereo,  and  are  too  neatly  organized  to  be  a  bunch  of  personnel. 

At  (9),  the  objects  are  not  vehicles  because  the  "road"  shows  only  trail 
traffic.  There  is  a  severe  road  cut  at  the  upper  1/3  of  the  annotation 
and  the  outline  of  the  objects  is  fuzzy  compared  to  the  sharp  outline  of 
the  graves  at  (l).  The  objects  at  (9)  are  bushes  commonly  found  on  the 
broader,  higher  dikes. 


Th'-  items  at  (1)  are  not  weapons  positions  but  rather  grave  mounds 
similar  to  those  in  the  upper  stereo  pair,  and  the  same  indicators 
apply.  This  is  another  example  of  the  serious  error  of  calling  a  grave 
a  weapons  position. 

The  shrines  at  (2)  are  larger  and  more  elaborately  constructed  than  most 
graves  but  lack  the  trackage,  field  of  fire,  or  strategic  location  of  a 
weapons  position. 

Tree  shadows  and  logs  such  as  at  (3)  ate  often  mistaken  for  sampans  but 
lack  the  size  and  clarity  of  outline  of  a  true  sampan.  Usually,  the 
tree  casting  the  shadow  can  be  found. 

At  (4)  are  personnel  distinguished  by  their  size  and  location  on  trails 
or  in  the  fields. 

The  sampans  at  (5)  are  clearly  sampans,  in  contrast  to  those  pseudo¬ 
sampans  at  (3),  but  have  nevertheless  been  omitted  in  the  reports  of 
some  interpreters.  Note  that  sampans  do  occur  on  land  where  they  are 
pulled  up  for  maintenance  or  storage. 

The  objects  at  (6)  are  neither  vehicles  nor  personnel.  By  comparison 
with  the  houses  nearby,  they  are  too  small  to  be  other  than  little  carts, 
but  too  big  and  blocky  to  be  personnel.  Noting  the  similar  regularly 
spaced  pattern  at  the  top  side  of  the  same  paddy  confirms  their  identifi¬ 
cation  as  strawstacks. 

The  items  at '(7)  are  too  tall  to  be  vehicles  and  too  blocky  and  lacking 
in  movement  to  be  personnel.  They  are  rather  neatly  aligned  to  be 
bushes  and  rather  tall  and  spindly  to  be  strawstacks.  Perhaps  they  had 
best  be  called  Unidentified  objects  (lIIOfs). 
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Top  Left  Figure 

This  photo,  at  1:8000  scale,  illustrates  graves  and  craters,  two  types 
of  circular  objects  which  inexperienced  interpreters  often  call  foxholes 
and  weapons  positions.  At  (1),  the  quantity  and  highly  random  arrange¬ 
ment  of  the  'revetted'  walled  entrances  protruding  from  the  circle  is  a 
prominent  feature  with  this  type  of  grave. 

At  (2),  the  random  size  and  arrangement,  spoil,  and  lack  of  symmetry 
identify  these  objects  as  craters.  When  weapons  positions  or  foxholes 
do  occur  in  and  around  cratered  areas,  great  care  must  be  taken  to 
distinguish  between  the  two  types  of  holes  in  the  ground. 

Vietnamese  trails  (5)  often  look  very  much  like  roads  and  do  fulfill 
many  of  the  functions  of  roads,  even  though  much  narrower.  However, 
their  carrying  capacity  is  limited  and  if  considered  a  road  and  used  as 
an  indicator  of  scale,  gross  errors  may  result.  Compare  these  trails 
with  the  tank  tracks  in  upper  center. 

Shrines  such  as  at  (4)  are  found  infrequently.  They  are  distinguished 
from  fortifications  by  the  lack  of  defenses  other  than  the  walls  them¬ 
selves,  and  the  absence  of  military  clutter. 

Annotation  (5)  indicates  personnel  on  a  trail.  Note  that  if  the  trail 
was  erroneously  considered  a  road,  the  personnel  might  be  misidentified 
as  vehicles. 

Top  Right  Figure 

Graves  such  as  those  at  (1)  may  occur  in  large  or  small  groups.  In 
large  groups  they  are  readily  distinguished  from  military  positions  by 
their  random  pattern,  even  if  individual  graves  do  look  like  AA,  AW,  or 
artillery  positions.  In  smaller  groups,  proper  identification  may  take 
more  care,  and  even  masses  of  graves  should  be  checked  for  military 
positions  which  may  be  hidden  amongst  them. 

Size  is  the  most  important  factor  in  identifying  annotation  (2)  as  a 
trail.  Compare  its  .  rdth  with  the  width  of  a  dike  in  a  rice  paddy; 
such  dikes  are  rarely  more  r.han  24''  wide. 

Size  is  also  the  most  significant  feature  in  identifying  the  brush  and/ 
or  grass  hummocks  at  (5).  After  the  trail  had  been  identified  as  a  road 
by  several  inexperienced  interpreters,  these  objects  were  called  a 
convoy  of  vehicles  along  the  road,  when  actually  they  are  less  than  two 
feet  wide. 


In  this  illustration  are  numerous  craters  which  have  been  misinterpreted 
as  other  military  activity.  At  (1),  air-burst  bombs  have  left  large 
circular  scars  with  scalloped  edges  and  no  noticeable  pit.  Particularly 
in  more  heavily  vegetated  areas,  such  craters  have  been  mistaken  for 
assembly  points  or  supply  points  when  the  radial  marks  of  the  thrown-out 
spoil  were  thought  to  be  trails. 


At  (2)  we  have  shell  craters  which  are  distinctive  because  of  their 
appearance  like  an  edge  view  of  a  flying  saucer.  This  is  because  the 
main  bursting  force  goes  to  either  side  with  a  secondary  force  forward 
and  very  little  force  extending  back  toward  the  weapon.  These  shell 
holes  are  misidentified  as  foxholes  or  weapons  positions  when  the  lateral 
extensions  are  thought  to  be  access  trails. 


Comparison  with  the  tank  tracks  at  (4)  or  the  shell  holes  and  bomb  craters 
Indicates  that  the  trace  at  (3)  is  far  too  narrow  to  be  a  road  and  thus 
must  be  a  trail. 


Bottom  Right  Fis 


Many  civilian  features  are  misidentified  as  military. 

The  round-walled  graves  at  (1)  look  very  much  like  AA  positions  but  lack 
the  proper  arrangement,  spacing,  communications  trenches,  ammo  revetments, 
and  fire  control  center  which  would  be  associated  with  a  true  AA  position. 

Rectangular  or  square  walled  shrine/graves  such  as  those  at  (2)  may  look 
like  fortifications  but  have  their  own  characteristic  appearance,  differ¬ 
ent  from  the  characteristic  military  installation  with  its  firing  bays, 
trenches,  supplies,  strategic  location,  etc. 

At  (3)  is  a  temple  complex  which  has  the  appearance  of  parade  grounds, 
barracks,  and  other  components  of  a  military  installation.  However,  the 
manicured  gardens,  sculptured  ponds,  complete  absence  of  defenses  or 
military  clutter,  and  the  large  number  of  graves  surrounding  the  complex 
indicate  its  non-military  nature. 

The  clumps  of  bushes  at  (4)  are  not  vehicles.  Look  closely  to  note  the 
individual  bushes  and  do  not  be  misled  by  the  length-width  ratio  and 
square  corners  into  thinking  that  they  are  vehicles.  Note  that  these 
objects  leave  no  trackage. 

The  drainage  ditch  or  ravine  at  (6)  is  not  a  trench.  It  is  too  broad, 
irregular  in  outline,  and  mottled  in  texture.  Compare  with  (5)  which 
is  a  trench. 
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APPENDIX  C: 


SEA  TARGET  LIST 


WEAPONS  POSITIONS  -  Designate  type  if  possible,  including  AA,  AW, 

artillery,  mortar,  etc.  and  indicate  whether  or 
not  occupied. 

BUNKERS  -  Buildings  of  heavy  construction  or  otherwise  for¬ 

tified,  not  to  include  rooms  excavated  into 
hillsides  or  fortified  weapons  positions. 


TUNNEL  ENTRANCES 
TRENCHES 

FOOT/BICYCLE  BRIDGES 


-  Includes  rooms  dug  into  hillsides. 

-  Include  only  military  trenches  (firing  or  commu¬ 
nication),  not  canals  or  irrigation  ditches. 

-  Do  not  report  vehicle  bridges. 


ROAD  BLOCKS 


-  Built-up  barricades  only;  do  not  include  road  cuts. 


STRONG  POINTS 

TRIANGULAR  FORT 

PERSONNEL 


-  Indicate  number  of  weapons  positions  and  whether 
or  not  occupied. 

-  Indicate  condition--occupied,  unoccupied,  aban¬ 
doned,  etc. 

-  Indicate  quantity  and  what  they  are  doing. 


SAMPANS 
VEHICLho 
SUPPLY  POINT 


-  Designate  motorized  or  non-motorized,  if  possible. 

-  Indicate  type — ox-carts,  trucks,  pedicabs,  etc. 

-  Includes  both  storage  and  distribution/trans- 
shipment  types.  Indicate  occupied/unoccupied. 


Fill  in  the  following  information  on  the  bottom  portion  of  your  answer 
sheet.  It  is  not  necessary  to  make  annotations  for  these  items. 


FOXHOLES 

ROAD  CUTS 


TRAILS 


CONFIDENCE 


-  Estimate  the  number  on  the  entire  frame,  includ¬ 
ing  overlaps  with  previous  frames. 

-  Count  the  number  of  sections  of  road  containing 
road  cuts  of  any  type.  Do  not  count  individual 
pits  or  ditches. 

-  Estimate  the  number  of  trails  and  indicate  their 
apparent  usage  —  to  and  from  fields,  between  vil¬ 
lages,  no  apparent  reason  for  existence. 

-  C  “  Certain 
Prob  *  Probable 
Poss  “  Possible 
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APPENDIX  D 


INTERPRETATION  REPORT  FORM 


MISSION  NO.  _ 

FRAME  NO.  _  II  NAME 


1 

2 

3 

4 

5 

D 

IDENTIFICATION 

QTY 

CONF 

REMARKS 

■ 

■ 

■ 

| 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

m 

m 

m 

m 

m 

■ 

■ 

■ 

■ 

■ 

■ 

Roadcuts 

m 

SSI 

Trails 

S3 

ki 

Foxholes 

3><^ 

m 

PREVIOUS  EDITIONS  OF  THIS  FORM  ARE  OBSOLETE 
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ANALYSIS  OF  VARIANCE  FOR  THREE  DEPENDENT 
VARIABLES  IN  EXPERIMENT 


Source  of  Variation 

Sum  of  Squares 

DF 

MSS 

F 

RIGHTS 

Between  Groups 

610.63 

1 

610.63 

8.104* 

Person  Within  Group 

9041.80 

120 

75*35 

9652.43 

121 

Within: 

Keys 

1475.41 

1 

1475.41 

41.549* 

Missions 

844.73 

1 

844.73 

23.789* 

Error 

4261.36 

120 

35.51 

658I.5O 

122 

Total 

16,233.92 

243 

WRONGS 

Between  Groups 

579-41 

1 

579-41 

1.614 

Person  Within  Group 

43,069.05 

120 

43,648.46 

121 

Within: 

Keys 

2124.59 

1 

2124.59 

17.906* 

Missions 

2124.59 

1 

2124.59 

17.906* 

Error 

14,237.82 

120 

118.65 

18,487.00 

122 

Total 

62, 135.46 

243 

ACCURACY 

Between  Groups 

443. 61 

1 

443. §1 

0.741 

Person  Within  Group 

120 

598.79 

72,298.79 

121 

Within: 

Keys 

12,393.92 

1 

12,393.92 

69.656* 

Missions 

626.56 

1 

626-56 

3.521 

Error 

2i.j22i.-Sl 

120 

177.93 

34,372.00 

122 

Total 

106,670.79 

243 

*F.05(  1,120)  =  3.92. 
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